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Abstract. 

In the context of the p-spin spherical model, we introduce a method for the computation of the number 
of stationary points of any nature (minima, saddles, etc.) of the TAP free energy. In doing this we clarify 
the ambiguities related to the approximations usually adopted in the standard calculations of the number 
of states in mean field spin glass models. 



PACS number: 75.10.N, 05.20, 64.60.C 



A Cavagna, I Giardina and G Parisi 

1. Introduction. 

Mean field spin glass models are characterized in their low temperature phase by the great number 
of metastable as well as equilibrium states. A question which naturally arises in this context is the 
computation of the number Af of these states, or, more precisely, the analysis of how this number increases 
with the size N of the system. 

In models with a continuous transition, as the SK model [1], the equilibrium thermodynamics is 
dominated by a number of states that remains finite when N — > oo, while there is an exponentially high 
number of metastable states [2], which do not contribute to the thermodynamics of the system. On 
the other hand, models with a discontinuous transition, as the p-spin spherical model [3] [4] [5], exhibit 
a temperature range where the number of metastable and equilibrium states with given energy density 
E grows exponentially, i.e Af(E) <~ exp(iVS(S)) [6]. In this last case the knowledge of the complexity 
T,(E) is crucial, since it gives a finite entropic contribution to the global free energy [7]. It is therefore 
particularly important in this case to have a well defined method to compute the number of states of the 
system. 

The standard strategy to perform this calculation is grounded on the formulation of mean field equa- 
tions for the local magnetizations, the TAP equations [8]. The solutions of these equations are identified 
with equilibrium or metastable states of the system and therefore one simply resorts to count the number 
of these solutions. 

This standard approach contains however some ambiguities. The TAP solutions can be viewed as the 
stationary points of a TAP free energy /tap, function of the magnetizations [2] [6] [7], therefore only the 
minima of this free energy can actually be identified with metastable or equilibrium states of the system. 
Yet, there are surely many other kinds of stationary points different from minima. When in the standard 
approach one counts the number of TAP solutions it is not clear whether only the genuine states of the 
system are taken into consideration. 

Moreover, a typical approximation of the standard method is related to the modulus of the determinant 
of the free energy Hessian (i.e. the Jacobian of the equations), which appears in the integral over all the 
solutions [9]. The presence of this modulus is fundamental to avoid a trivial result: if one tries to count 
the number of stationary points of a function without this modulus, each stationary point is weighted with 
the sign of the Hessian and one obtains a simple topological constant, by virtue of the Morse theorem [10]. 
Nonetheless, in the standard approach this modulus is always disregarded to simplify the computation. 

From what said above we are leaded to say that the standard procedure is not really under control. 
Nonetheless, at least in the case of the p-spin spherical model, this standard calculation gives a result [6], 
that has been exactly confirmed by a completely different approach [11]. This result is therefore correct, 
although all the approximations involved are not well justified. On the other hand for the case of the SK 
model there is no confirmation of the standard result of [2] . 

The aim of this paper is to clarify this subject, at least in the case of the p-spin spherical model. 
In the context of the replica approach we show that different solutions of the saddle point equations for 
the overlap matrix are related to different kinds of stationary points (minima, saddles, etc.). Grouping 
them into classes characterized by the number k of their instable directions, we find that each class has 
a different complexity (E) . By virtue of this result we are able to extract separately from the total 
number of solutions the contribution of minima and of saddles of various indices k, discovering that there 
is an ordering of the complexities Efe(S): at a given energy E only one kind of stationary points is 
exponentially dominant over all the others, so that in the thermodynamic limit the weight of the sign 
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of the determinant has no influence. Therefore, as long as the energy is kept fixed, the modulus can be 
disregarded and the standard approach gives the correct result. 

2. The complexity. 

The p-spin spherical model is defined by the Hamiltonian 

ii<...<i p 

The spins s are real variables satisfying the spherical constraint J2i s i — ^7 where N is the size of the 
system. The couplings J are Gaussian variables with zero mean and variance p\/2N p ~ 1 . In the context 
of the TAP approach [8], one formulates a set of mean field equations for the local magnetizations 
Tij = (si)- In [7] it has been introduced a free energy density /tap, function of the magnetizations m^. 
The minimization of frAP with respect to m t gives the TAP equations of the system. We can express 
the magnetization vector m in terms of its angular part a and of its self-overlap q = l/^V^ i m; 2 : 

m, = V? o-i ; a ■ a = ^ a\ = N . (2.2) 

i 

The TAP equations for a read [7] 

= -p J2 Jl,i 2 ...i p o- i2 ...a ip -pEai =%{a-E) , l = l,...,N (2.3) 

i 2 <...<i p 

where E is the zero temperature energy density 

E = -^ E J --> n • ( 2 - 4 ) 

H<...<i p 

In the following we shall always refer to the zero temperature energy density. The equations for a do 
not depend on the temperature, while the equation for q does [7]. Moreover the q equation has solution 
as long as the energy density is lower than a maximum value of the energy, called threshold energy E t h- 
The dependence on temperature of the set of TAP solutions {m{T)} a=1 ,,,^f comes entirely from q, while 
their multiplicity Af is encoded in equations (2.3) and thus does not depend on the temperature. It turns 
out that there is an exponentially high number of solutions of (2.3) for each given value of the energy 
density E, N{E) <~ exp(NY,(E)), where T,(E) is the complexity, computed for this model in [6]. T,(E) is 
an increasing function of E, which reaches a finite value for E = E t h- To avoid any confusion, we note 
that the TAP free energy density of a solution at temperature T is unambiguously determined by its zero 
temperature energy density E. Therefore in the following we shall always use E to label TAP solutions. 

We start our analysis with the computation of T,{E), paying special attention to the nature of the 
stationary points actually considered. By definition we write 

£(25) d = f lim 1 \^N{E) (2.5) 

N—>oo JV 

We average the logarithm of Af since this is the extensive quantity. To perform this average it is necessary 
to introduce replicas already at this level of the calculation. However, it can be shown that the correct 
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ansatz for the overlap matrix is symmetric and diagonal and this is equivalent to average directly the 
number Af of the solutions. Therefore we will perform the annealed computation: 

1 



E(£) = lim -logM(E) . (2.6) 

N—>oo iv 



In terms of the angular parts (2.2) we have 



1 f N 
E(£) = lim — log / Va 5{a ■ a - N)T\ 5{%{a;E)) \detH(a;E)\ (2.7) 

J 1 = 1 

where H(a;E) is the Hessian of the TAP equations evaluated in the solution a of energy density E. It 
is given by 

H r ,l(v, E) = = -Pip - 1) E J r,l,is-i p °i P - PE5 r ,l ■ (2.8) 

i 3 <...<i p 

We stress that by means of formula (2.7) we are counting only the solutions with a given energy density 
E. This is a crucial point: the principal effort of our discussion will be to show that, as long as E < E t h, 
if we keep the energy fixed the modulus in (2.7) can be dropped without affecting the result in the limit 
N — > oo. Wc shall return on this point with greater details at the end of our discussion. We therefore 
perform the calculation without the modulus, showing a posteriori which are the justifications of this 
procedure. Let us introduce a Bosonic representation both for the determinant and the delta functions 
that implement the TAP equations: 

detH= lim{detW}~ f = lim f V<f> a exp \ -- (4> a H<p a ) ) 



a=l 



N 

II 5 (Ti{^E)) = / 2? M exp(z M T) 
1=1 J 



(2.9) 



where the sums over repeated site indices are understood. The average over the disorder generates 
couplings between the fields <f>, a and /i. A crucial approximation is to set equal to zero the couplings 
4> a ■ a and cf) a ■ fj, which depend on one replica index and which break the rotational invariance in the space 
of the replicas. We will see that this approximation is consistent with all the solutions we shall consider 
for the saddle point equations. Thus we retain only the terms <j> a ■ <j>b, M ■ M an d M " a - It is easy to see 
that this approximation is equivalent to write 



i r N 

E(£) = lim — log / Va 5(a ■ a - N) |7 5{Ti(a;E)) x detH(a;E). (2.10) 

J 1 = 1 

Once averaged over the disorder, because of the spherical constraint the part of the determinant does not 
depend on a any more. Therefore we have 

E(£) = A(E) + B(E) (2.11) 



A(E) = lim log J Va 5(a ■ a - N) J V/j,ex.p(i/j,T) 



B(E) = Jim log J Va 5(a -a- N) lim^ V(j) a exp y~ ^(^ a W^°) 



(2.12) 
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The first integral does not involve replicas and gives the following contribution 

A{E)= l -- l -\og P --E 2 . (2.13) 

The second integral is more subtle to solve because it contains replicas and an appropriate ansatz has to 
be chosen to solve the saddle point equations. Moreover this integral is the one related to the Hessian 
of the TAP solutions and thus it contains information on the nature of the solutions (minima, saddles 
or maxima) that we are counting. Once averaged over the disorder and introduced the overlap matrix 
Qab = -pip - l)(0 a • <p b )/2N, we obtain 

B{E) = aJ™ 7f l0g ^2/ ^expj-iV (^T) +l^gdet(-pE + Q)^ . (2.14) 

As an ansatz for the matrix Q we take Q a b — Qa Sab- In this way the exponent of (2.14) splits into n 
independent parts, each one giving the same saddle point equation for q a , whose possible solutions are 



<la=q± = P , (E±y[W^E*\ , E th = -J 2 -^-^. (2.15) 



2 V V "7 ' *" y p 

We restrict our discussion to E < E t h, so that q± are real. Since each q a can assume one of these two 
values, we have a multiplicity of different solutions. The analysis of the fluctuations shows that there is 
a stable solution So, given by 

S : q a = q+ , a = 1, . . . ,n (2-16) 

This solution is invariant under rotations in the replica space and thus the approximation we made setting 
to zero the terms depending on one replica index turns out to be consistent. The solution So gives the 
complexity 

Z (E) = + \og(- P E + q+ ) + A(E) (2.17) 

with A given in (2.13). This is the known result of [6]. It is important to note that this result has been 
confirmed in the analysis of [11] and [12] where, by means of a completely different method, it has been 
shown that So is equal to the logarithm of the number of genuine states of the system, and thus that So 
is the complexity of the minima of the TAP free energy. 

Nonetheless, we note the presence of many other solutions of the saddle point equations, involving 
both the values q± . In particular we are interested in the solution Si with the lowest degree of instability, 
that is 

Si: qi = q- , q a = q+ , = 2, ...,n (2.18) 

(and permutations). This solution presents a one step breaking of the rotational invariance in the replica 
space. Therefore one can concern about the fact that we have disregarded terms breaking this invariance. 
To check this point, we have performed the whole computation retaining the terms (f) a ■ a, <f) a ■ fi and we 
have looked for a solution breaking the rotational invariance in the replica space. We found analytically 
that the saddle point equations give as a unique solution <fi a ■ a — and <j> a ■ /i = and thus that solution 
Si is recovered. The complexity Si arising from Si is 

ME) = - + I lo g (- P E + q+)-\ W-pE + q-) + A(E) (2.19) 

which is lower than S , since \q_\ > \q+\. In this context it is not clear which is the physical meaning 
of the complexity Si, neither if there is one. Moreover, apart from the fact that the complexity So is 
confirmed by a different method to be related to the number of minima, we have given no justification of 
dropping the modulus in the original formula. We shall see in the next sections that the analysis of the 
average spectrum of the TAP Hessian gives an answer to both these questions. 



- 5 - 



A Cavagna, I Giardina and G Parisi 



3. The Hessian spectrum. 

In the previous section we made the approximation of setting to zero the couplings <fi a ■ a and <j> a ■ p. We 
stress that this approximation is consistent when considering the solutions Sq and Si . As a consequence, 
what appears in expression (2.10) is the Hessian function evaluated in a generic vector a, and not in a 
TAP solution. This means that, in the context on this approximation, the properties of the TAP Hessian 
that are relevant in determining the behaviour of S, are well encoded in the matrix 7i(cr; E) which has 
the same functional form of the TAP Hessian, but requires a only to satisfy the spherical constraint. The 
average spectrum is then defined in the following way 



p(X;E)= lim f Va 5(a ■ <r — N) p.j(X; a) (3.1) 
where pj (A; a) is the spectrum for a given realization of the disorder whose expression is: 

p J (X;a) = -^-lmTT(n-X + ie)- 1 . (3.2) 

JV7T 

We can write the trace in the following way: 

N 



Tr(W - A + te)- 1 = - A + ie)" 1 ],, = 



i=i 



= lim J V(j) a (j) 1 ■ (j) 1 exp j-1 a ( n - X + ie )^ 



(3.3) 



Once averaged over the disorder J and exploited the spherical constraint on a, this computation becomes 
analogous to the one of the average spectrum of a Gaussian ensemble of symmetric random matrices [13]. 
If we introduce the overlap matrix Q a b = —p(p — l)((f> a ■ 4>b)/2N we finally get 



p(X; E) = lim Im lim f VQ ab (-pE - X + Q) n : 



- 1 X 



/ Tr Q 2 1 ^ (3 ' 4) 

cxp^-A ^ — j± + - logdet(-pE - A + Q) 



»)} 



It is important to note the great similarity between equations (3.4) and (2.14). If we choose once again 
a diagonal ansatz Q a b = w a S abj we get the following solutions of the saddle point equations 

Wa = w±{X ) = l ( ± +E ±<l[- + E) -Ef„ ] 




2 \p 

where E t h is the same as in equation (2.15). For A = the integrand in (3.4) is identical to the one of 
(2.14) and w±(0) = q±. As in the case of the complexity we have a multiplicity of different solutions. To 
get a finite contribution to p it is necessary that the argument of the exponential in (3.4) is zero. Since 
n — > 0, this can be achieved taking the same value for each w a - Moreover, the condition p > shows 
that we must take the solution 

S : w a = w + (X) , o=l,...,n (3.6) 
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that is exactly the same kind of solution that leaded to S . If we look at (3.5) we can see that the 
non-zero contribution to p conies from the region — pE + pE t h < A < — pE — pE t h 1 where w + develops 
an imaginary part. Thus 

Po(A; E) = npJp~^rj^ p2E * h ~ {x+ pE)2 ■ (3J) 

We stress that the solution So is the only one that gives a finite contribution po to p. Formula (3.7) is the 
well known Wigner semicircle law [14], that can be obtained for symmetric Gaussian random matrices 
also without using replicas [13]. This result tells us that for E < E t h the averaged spectrum has a strictly 
positive support and thus the typical determinant of the Hessian is positive, i.e. that the dominant part 
of TAP solutions with energy density E < E t h are minima. On the other hand, when E approaches E t h 
the lowest eigenvalue A = p(E t h — E) goes to zero. Therefore the typical solutions with E = E t h have 
some flat directions [15]. 

We understand now the reason why the complexity of (2.17) is related to the number of minima: the 
solution So of the saddle point equations leading to S is exactly the same as the one leading to the 
eigenvalue distribution po, which has positive support. 

The important thing is that in this context it is possible to give a precise physical interpretation of 
the solution Si of (2.18): as we are going to show in the next section, Si is related to the exponentially 
small corrections to the distribution p and therefore gives informations on those TAP solutions which 
are not minima. 



4. Exponential tails and complexity of the saddles. 

For an ensemble of symmetric random matrices with a Gaussian distribution it is possible to compute 
corrections to the semicircle law, when TV is large but finite. In particular, it is possible to compute the 
correction to the averaged spectrum related to the probability of having a single eigenvalue outside the 
semicircle support. 

In the context of our calculation this can be achieved by considering solutions of the saddle point 
equations for p different from Sq. In particular, we are interested in corrections to po in the eigenvalue 
region on the left of the semicircle region, i.e. for A < —pE+pE t h, since this tail contains the contribution 
of the negative eigenvalues. In this region we consider the solution Si 

Si: wi-w^(X) , w a = w + (\) , a = 2, ...,n (4-1) 

(and permutations); from equation (3.4) we get 

Pi{\,E) =r(X,E) e - NA< - x ' E ) , A(A,£) >0 for \<- P E+pE th (4.2) 

which goes exponentially to zero as N — > oo. In the computation of pi a crucial role is played by the 
fluctuations around the saddle point solution Si, since the fluctuations matrix has an instable direction 
which provides the imaginary part necessary for pi to be non-zero outside the semi-circle. On can easily 
check that both r(A, E) and A(A, E) coincide with the expressions obtained for the Gaussian random 
matrices with other methods [13]. This is therefore a correct result. The important quantity for our 
analysis is A(A, E) 

A ^ E) -2p1p^ ) -2p1p^) + 2^{ -x4e + W+ ) • (43) 
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Solution Si then gives the exponentially vanishing left tail, due to the probability of having one eigenvalue 
outside the semicircle. Since this tail is different from zero also in the negative semi-axis, we can calculate 
the probability of having a negative eigenvalue, i.e. the exponentially small probability of finding a TAP 
solution which is a saddle with one negative eigenvalue and has energy density E. This probability is 

P(-)= f° d\p 1 (X,E)^ e ~ NA (°> E ) ; iV^oo (4.4) 

J — oo 

In this context solution Si has a clear physical interpretation: it is related to the contribution of TAP 
saddles with one negative eigenvalue, in the energy range E < E t h- Given this, we can try to push 
further this interpretation. As we have seen in section 2, the same solution S\ gives rise to a complexity 
Si smaller than So, whose meaning was not clear. Now we can make the hypothesis that Si is the 
complexity of the saddles with one negative eigenvalue. To prove this statement we note that once we 
have the number Afi(E) ~ exp(NHi(E)) of saddles with one negative eigenvalue and energy density E, 
we can easily compute the probability P(_) of having one of these saddles 



_ MCg) _ ejVSl(£) _ -JV[Eo(S)-Ei(B)] (4 ^ 



where we used the relation So(-E) > Si(S). From a comparison between (4.5) and (4.4) we see that it 
must hold 

A(0,£) = S (£)-Si(£) (4.6) 

It is not difficult to see from equations (2.17), (2.19) and (4.3) that this equation is fulfilled. Our 
hypothesis is therefore correct and we can then write: 

1 



Si(£)= hm -IogM(^) (4.7) 

N—>oo iv 

where, as already said, Afi(E) is the number of TAP solutions of energy density E, which are saddles 
with one negative eigenvalue. This result can be generalized. If we consider the following solution Sk of 
the saddle point equations for S 

Sk- q a = q- , a = 1, . . . , k , q a = q + , a = k + l,...,n (4.8) 

(and permutations), we obtain from (2.14) the complexity 

It is not a surprise the fact that S^ is related to the number of TAP solutions which are saddles with k 
negative eigenvalues. Indeed the probability of finding such a solution is 

^-) = N fc ~^ W) (4-10) 
so that to prove our assertion it is sufficient to verify that holds the relation 

k A(0, E) = So(-E) — S fe (_E) (4.11) 

as it does. In writing equation (4.10) we can disregard the correlations between different negative ei- 
genvalues, as long as k is much smaller than N. We conclude that, as a general result, T,k{E) is the 
complexity of TAP saddles with k negative eigenvalues and energy density E. 
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Since \q_\ > \q+\ we have that £ (£) > ^i(E) > ... > > E fc+ i (£).... Thus all the TAP 

solutions, also those with some negative eigenvalues, are exponentially numerous in TV. Nevertheless, the 
number of minima is exponentially higher than the number of saddles with one negative eigenvalue, which 
is exponentially higher than the number of saddles with two negative eigenvalues, and so on. This is the 
very reason why, as long as E < E t h, the approximation of dropping the modulus in (2.7) is justified. In 
Figure 1 we have plotted So, £1 and £ 2 as a function of E. 



0.015 



0.01 



0.005 




-1.165 



1.155 



Figure 1: The complexity So of the TAP minima (solid line) and the complexities Si and £2 of the TAP saddles with one 
and two negative eigenvalues (respectively dotted and chain line), as a function of the zero temperature energy density E. 
The threshold energy is E th =— 1.1547 for p=3. The minimum saddles energy is Eo=— 1.1688. 



From equation (4.9) we note that T,k(E t h) = T*o(E t h), for each k, since q+ = q- at the threshold energy. 
This equality is very important. If we try to count the total number of solutions neglecting the modulus 
in equation (2.7), a trivial result is obtained [9], since we are weighting each stationary point with the 
sign of the determinant (this is the Morse theorem). This is the reason why we considered solutions 
with a given fixed energy E. Yet, for what said above, if we integrate our result over all the energies E, 
we must recover the result predicted by the Morse theorem. Remembering that the q part of the TAP 
equations admits solutions only for E < E t h, we have from our calculations: 



I alE jVa 6(a ■ a - N)f[ 6(Ti(a;E)) dct W(er; £) 
^ 1=1 



(4.12) 



a 



e N^{E th ) + ^ e N^(E th ) + ^ e N^(E th ) + _ 



In this formula we must introduce all the 's coming from all the solutions of the saddle point equations 
for S, which refer to stationary points of any nature. One can easily see that all the X^'s are monotonously 
increasing functions of E which reach their maximum value at E t h, so that we can substitute the integral 
in (4.12) with the maximum of the integrand. The prefactors a , 01, . . . come from the fluctuations around 
each saddle point solution and contain the sign of the determinant. It is exactly the combination of these 
signs that gives rise to the Morse theorem. From equation (4.12) is then clear that a necessary condition 
to get a trivial topological constant is that T, (E t h) = T,i(E t h) — T, 2 {E t h) = . . ., so that we can sum all 
the terms on the same foot. As said above, this necessary condition is fulfilled by our calculation. 
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Besides, from equation (4.12) it is finally clear what is the role of the modulus in the calculation: taking 
| det 7Y| is equivalent to take the absolute value of the prefactors at, thus preventing from obtaining a 
trivial result. Yet, at fixed energy E < E th , one of the terms exp(NT, k (E)) is always bigger than all the 
others and therefore in the limit N — > oo the signs of the prefactors a k have no influence on the final 
result. As we have said, this dominant term turns out to be the one with k = 0, which gives exactly the 
contribution of the minima. 

We note that it should be possible to show that is related to the number of the saddles with k 
negative eigenvalue directly from (2.14). If we keep n finite this integral is equivalent to (det W)™. Taking 
the saddle point solution S k and appropriately computing the Gaussian fluctuations around it, it should 
be possible to single out a factor (— l) hn related to the sign of the determinant. Unfortunately, we did 
not succeed in performing this quite complex computation. 

From figure 1 we see that there is a minimum energy density Eq below which no saddles with finite 
complexity are found. Therefore when considering a state with energy density E < E , the value 
AE = Eq — E is a lower bound for the energy density barrier between this state and any other state of 
the system. In [12] a potential function has been introduced, whose minima are by construction equivalent 
to metastable or equilibrium states of the system. With this method it has therefore been possible to give 
an estimate for the barriers separating two states [16]. It turns out that this estimate is fully consistent 
with the result of the present work. 

5. Conclusions. 

The main result of this paper concerns the organization of the stationary points of the TAP free energy in 
the p-spin spherical model. If we classify these points according to the number k of negative eigenvalues of 
their Hessian, we find that each class is characterized by a complexity T, k (E) which gives the exponentially 
high number of TAP solutions of energy E in that class, N k {E) <~ exp(NT, k (E)). In the energy range 
E < E t h we find that T, k (E) > T, k+ i(E) for each value of k. This means that in this energy range minima 
are exponentially dominant in number over all the other stationary points. 

From what said above we conclude two things: First, if we compute, even in the most rigorous way, 
the complexity S(-E) at a given fixed energy, according to formula (2.5), we automatically recover T, (E), 
i.e. the complexity of the minima. Secondly, the modulus of the determinant simply contributes to the 
sign of the prefactor of the dominant contribution, since at fixed energy all the other terms are vanishing 
in the thermodynamic limit. Therefore, when such a structure of the stationary points is present it is clear 
that the naive calculations which do not discriminate among minima, saddles, etc. and which disregard 
the modulus are, notwithstanding this, consistent [6] [17]. 

We stress that it is crucial to keep the energy fixed in the calculation, but more important is the 
fact that all the complexities arc different, so that only one of them survives in the limit N — > oo. This 
becomes clear when E is equal to the threshold energy E t h- here all the SVs are equal and a trivial result 
is recovered. 

From a technical point of view we note that the use of a Bosonic representation for the determinant 
and the consequent replica approach introduces a degree of arbitrariness in the choice of the saddle point 
solutions which makes it possible to extract the contributions of different classes of stationary points. 
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